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(54) 1 Apparatus and method for video frame rate conversion 



(57) A method and apparatus for converting 
between different video formats, which provide smooth 
video motion by eliminating unnatural effects which 
could be introduced in the process of video format con- 
version. A frame interpolator produces interpolated 
frames from a first video signal given in a first video for- 



mat by using motion vectors of the first video signal. . 
From the interpolated frames, a video signal generator 
produces a second video signal in a second video for- 
mat that is incompatible with the first video format. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0001] The present invention relates to an appara- 
tus and method for converting between different video 
formats, and more particularly, to an apparatus and 
method for converting between different video formats 
to enable visual communication between remote users 
of different television systems. 

2. Description of the Related Art 

[0002] There are three major standard television 
systems available today: NTSC, PAL, and SECAM. 
NTSC is used in the United States and Japan, while 
PAL and SECAM mainly in European nations. For com- 
parison, FIG. 18 shows some key specifications of 
those three different video standards, including: frame 
frequency, the number of effective picture elements 
(pels or pixels) per line, and the number of effective lines 
per frame. Although PAL and SECAM seem to share 
some common parameters, SECAM is actually incom- 
patible in operation with PAL. 

[0003] As noted above, different video formats are 
used in different groups of nations, and to solve this 
incompatibility problem in international visual communi- 
cations, it is necessary to introduce some standardized 
video coding formats. Typical video formats for such 
purposes are H.261 and H.263, recommendations from 
the International Telecommunications Union-Telecom- 
munications Standards Sector (ITU-T)- They provide 
two common video formats called "Common Intermedi- 
ate Format (CIF)" and "Quarter Common Intermediate 
Format (QCIF)." Some of their key specifications are 
shown in FIG. 19 to make a comparison in terms of the 
number of picture elements per line, the number of lines 
per frame, and frame frequency. In this table of FIG. 1 9, 
the symbol "Y" represents luminance signals, and "Cr" 
and "Cb" denote color difference signals. 
[0004] FIG. 20 shows a simplified system structure 
for international digital video communications between 
two countries using different television system stand- 
ards. This video communications system interconnects 
ah NTSC system 2 and a PAL system 4 by using CIF as 
a vehicle for relaying digital video signals via a commu- 
nications satellite 3. To convert the video format, a video 
signal processor 200a is disposed at the NTSC side, 
and another video signal processor 200b at the PAL 
side. The former processor 200a comprises a format 
converter 201a and a codec (coder/decoder) 202a, 
while the latter processor 200b comprises a format con- 
verter 201b and a codec 202b. Such video signal proc- 
essors 200a and 200b are implemented as part of 
videoconferencing stations or videophone terminals, for 
example. 



[0005] Consider that the NTSC system 2 attempts 
to send video information to the PAL system 4. The for- 
mat converter 201a in the video signal processor 200a 
converts incoming NTSC video signal into a CIF video 

5 stream, which conforms to the common format for com- 
munications. This CIF video stream is then encoded by 
the codec 202a and transmitted to the communications 
satellite 3 via a radio transmitter (not shown). The radio 
signal retransmitted by the communications satellite 3 

10 reaches a receiver (not shown) attached to the video 
signal processor 200b. The video signal, now in the 
form of an electrical signal, is supplied to the codec 
202b for video decoding, where the original CIF video 
stream is reconstructed. The format converter 201b 

/ 5 converts this CIF video stream to a PAL video signal for 
use in the PAL system 4. It would be understood that 
video signal transmission in the opposite direction (i.e., 
from PAL to NTSC) can be achieved in a similar fashion. 
[0006] As described above, conventional visual 

20 communications systems use the CIF as a common for- 
mat to transport video information to remote sites. That 
is, the sender encodes video signals after converting 
from its local format to the CIF format, and the receiver 
converts from the CIF format to its local format after 

25 decoding reception signals. In such international video 
communications, conventional systems resolve the dif- 
ference in frame frequencies by subsamplihg or dupli- 
cating video frames. More specifically, when a video 
signal having a certain frame frequency has to be con- 

30 verted to a lower frequency (e.g.. 30 Hz to 25 Hz), the 
source frames are subsampled, or decimated, at prede- 
termined intervals. On the other hand, when a video sig- 
nal has to be converted to a higher frequency, some 
frames are sent twice to adjust the frame frequency. 

35 [0007] However, the above-described video conver- 
sion techniques introduce some unnatural effects into 
objects' motion in a video, because of the discontinuity 
of frame pictures. Viewers may perceive the drop or 
duplication of frames as awkward motion of objects, 

40 particularly when the video contains rapid motion such 
as a soccer ball flying in a parabola. 

SUMMARY OF THE INVENTION 

45 [0008] Taking the above into consideration, an 
object of the present invention is to provide an appara- 
tus for converting between different video formats to 
enable visual communications between users of incom- 
patible television systems, which maintains the smooth- 
so ness of object motions by preventing unnatural effects 
from being introduced in the process of video format 
conversion. 

[0009] Further, another object of the present inven- 
tion is to provide a video format conversion method for 
55 visual communications between users of incompatible 
television systems, which maintains the smoothness of 
object motions by preventing unnatural effects from 
being introduced in the process oFvideo format conver- 
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sion. 

[0010] To accomplish the first object, according to 
the present invention, there is provided an apparatus for 
converting from a first video signal in a first video format 
to a second video signal in a second video format that is s 
incompatible with the first video format This apparatus 
comprises: a frame interpolator which produces interpo- 
lated frames from the first video signal by using motion 
vectors obtained therefrom; and a video signal genera- 
tor which produces the second video signal from the 10 
interpolated frames. 

[001 1 ] To accomplish the second object, according 
to the present invention, there is provided a method of 
converting from a f irst video signal in a first video format 
to a second video signal in a second video format that is is 
incompatible with the first video format. This method 
comprises the steps of: producing interpolated frames 
from the first video signal by using motion vectors 
obtained therefrom; and producing the second video 
signal from the interpolated frames. 20 
[001 2] The above and other objects, features and 
advantages of the present invention will become appar- 
ent from the following description when taken in con- 
junction with the accompanying drawings which 
illustrate preferred embodiments of the present inven- 25 
tion by way of example. 

BRIEF DESCRIPTION OF THE DRAWINGS 



[0013] 



30 



FIG. 1 is a conceptual view of a video format con- 
verter according to the present invention; 
FIG. 2 is a block diagram of a video codec; 
FIG. 3 is a block diagram of a frame interpolator 35 
according to a first embodiment of the present 
invention; 

FIG. 4 is a diagram showing the concept of forward- 
reference and backward-reference motion vectors; 
FIG. 5 is a block diagram of a video signal genera- 40 
tor of the first embodiment; 
FIG. 6 is a diagram which shows inputs and outputs 
of frame memories; 

FIGS; 7 and 8 are diagrams showing frame interpo- 
lation processes in the first embodiment; 45 
FIG. 9 is a diagram showing the relationships 
between two video streams in the first embodiment; 
FIG. 10 is a block diagram of a frame interpolator 
according to a second embodiment of the present 
invention; so 
FIG. 1 1 is a block diagram of a video signal gener- 
ator of the second embodiment; 
FIG. 12 is a diagram which shows inputs and out- 
puts of frame memories; 

FIG. 13 is a diagram showing such a situation ss 
• ; where the proposed converter encounters a scene 
change when producing interpolated frames in nor- 
mal mode; 



FIGS; 14 and 15 are diagrams showing frame inter- 
polation processes in the second embodiment; 
FIG. 16 is a diagram showing the relationships 
between two video streams in the second embodi- 
ment; 

FIG. 17 is a flowchart showing a process of video 
format conversion according to the present inven- 
tion; 

FIG. 18 is a table showing some key specifications 
of NTSC, PAL, and SECAM; 
FIG. 19 is a table showing some key specifications 
of CIF and QCIF; and 

FIG. 20 is a diagram showing a simplified system 
structure for international digital video communica- 
tions between two countries using different televi- 
sion system standards. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

[0014] Preferred embodiments of the present inven- 
tion will be described below with reference to the 
accompanying drawings. 

[0015] FIG. 1 shows the concept of a video format 
converter according to the present invention. This pro- 
posed video format converter 1, comprising a frame 
interpolator 10 and a video signal generator 20. per- 
forms video format conversion to send visual informa- 
tion to a remote end that uses a different television 
system. Here, the term "video format conversion" refers 
to a process to convert frame frequencies. 
[001 6] According to the present invention, the frame 
interpolator 1 0 produces interpolated frames from a first 
video signal given in a first video format by using 
motion vectors obtained therefrom. (Technical details of 
frame interpolation will be described later.) From the 
interpolated frames, the video signal generator 20 pro- 
duces a second video signal in a second video format 
that is different from the first video format. 
[0017] Recall that conventional video format con- 
verters use simple frame subsampling or frame inser- 
tion techniques to convert from one frame frequency to 
the other. In contrast to this, the proposed video format 
converter 1 uses frame interpolation techniques to 
obtain a new frame frequency. That is, the video format 
converter 1 is designed to generate interpolated frames 
from the first video signal, and with these interpolated 
frames, produce the second video signal having a differ- 
ent frame frequency. It should be noted here that the 
proposed converter 1 produces interpolated frames in 
an adaptive manner, taking into consideration the differ- 
ence between consecutive frames, which is repre- 
sented by motion vectors. Because of the use of 
interpolated frames to recreate frames of the second 
video signal, the video format converter 1 of the present 
invention provides smooth motion in a scene, without 
awkward frame drops or other unnatural effects. 
[0018] A more specific -implementation of the 
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above-described video format converter 1 will be 
described below. FIG. 2 shows a structure of a video 
codec 100 which processes video signals by using 
internationally standardized video processing algo- 
rithms. This video codec 1 00 accepts video signals from 
two cameras, camera-A and camera- B, both having a 
frame frequency f1 , while its internal common video for- 
mat has another frame frequency 12. An external video 
monitor requires incoming video signals to have still 
another frame frequency 13. 

[0019] A video signal switching unit 110 is inter- 
posed between the cameras and video signal switching 
unit 110 to select either of the two video signals sup- 
plied from the camera-A and camera- B. The selected 
video signal having the first frame frequency 11 is 
applied to the video codec 100. Switching from camera- 
A to camera- B, or vice versa, will cause an instant and 
entire transition of screen images, which is referred to 
as a "scene change. " The video signal switching unit 
110 produces a scene change indication signal Ds to 
inform the video codec 100 of the occurrence of such a 
scene change. 

[0020] As the name implies, the video codec 100 
provides two major functions, video coding and decod- 
ing, employing two dedicated internal data paths for 
them. Referring to its functional blocks shown in FIG. 2, 
one data path running from left to right serves as a 
video coder, while the other data path running from right 
to left serves as a video decoder. The following section 
will explain the elements of each data path, as well as 
their operation. 

[0021] The video coding path begins with a video 
format converter 1-1 , which comprises a frame interpo- 
lator 10-1 and a video signal generator 20-1 . The frame 
interpolator 10-1 produces interpolated frames from a 
source video signal supplied from the video signal 
switching unit 110, by using motion vectors obtained 
from consecutive frames of the source video signal. 
With the interpolated frames produced, a video signal 
generator 20-1 produces another video signal stream 
having a frame frequency 12. When a scene change 
occurs, the video signal generator 20-1 receives the 
scene change indication signal Ds from the video signal 
switching unit 110, and based on this signal, it selects 
appropriate interpolated frames that are derived from 
correlated source frames, to continuously produce the 
converted video signal. 

[0022] The video signal produced by the video for- 
mat converter 1-1 is then supplied to a coder 101a to 
perform a data compression, or reduce its data size. 
More specifically, the coder 101a performs digital cosine 
transform (OCT) coding, quantization, and motion com- 
pensation, to compress spatial and temporal redundan- 
cies of video frames. After that, it applies variable-length 
coding to produce outgoing data coded in a prescribed 
format' The actual amount of the produced data may 
vary frame by frame, and hence the data rate. A trans- 
mission buffer 102a serves as temporary storage to 



smooth out such variations of the data rate. The buffer 
occupancy ratio is fed back to a coding controller 103, 
which controls the amount of coded data being pro- 
duced by the coder 101a, thus regulating the data flow 

5 to the transmission buffer 1 02a. At the final stage of the 
video coding path, a transmission coder 104a provides 
several functions necessary for data transmission. For 
instance, it inserts dummy bits when the transmission 
buffer 102a is empty, and it adds error correction code 

w to the coded bit stream to be transmitted. 

[0023] The video decoding path, on the other hand, 
begins with a transmission decoder 104b which 
receives a coded bit stream from a remote end. It 
removes dummy bits from the received bit stream, as 

is well as performs error check and correction. A reception 
buffer 102b regulates the flow of incoming data to 
ensure that each frame be decoded within a prescribed 
period. The received bit stream is then supplied to a 
decoder 101b, where a variable-length decoding algo- 

20 rithm is applied so as to extract each piece of coded 
data from the bit stream. The decoder 101b further exe- 
cutes dequantization, inverse-DCT, and motion-com- 
pensated image reconstruction, thereby expanding the 
coded data. 

25 [0024] The video signal reconstructed by the 
decoder 101b has a frame frequency 12 \ which is not 
compatible with the external video monitor. For fre- 
quency conversion, the video signal is supplied to a 
video format converter 1-2. which comprises a frame 

30 interpolator 1 0-2 and a video signal generator 20-2. The 
frame interpolator. 10-2. produces interpolated frames 
from the supplied video signal by using motion vectors. 
From these interpolated frames, the video signal gener- 
ator 20-2 produces a new sequence of frames, or a 

35 video signal with a different frame frequency, 13, which 
is compatible with the video monitor. 
[0025] The format of incoming coded bit stream 
may change from CIF to QCIF. or QCIF to CIF, during 
operation. When such a common format switchover is 

40 encountered, the video signal generator 20-2 selects 
appropriate interpolated frames that are derived from 
correlated source frames, to continue to produce the 
video signal with the frame frequency 13. 
[0026] A first embodiment of the present invention 

45 will now be described below with reference to FIGS. 3 to 
9. In this first embodiment, a video format converter 
reduces the frame frequency. For illustrative purposes, it 
is assumed here that the proposed converter is 
designed to make a frame frequency conversion from 

so 30 Hz to 25 Hz. 

[0027] Referring to a block diagram of FIG 3. a 
frame interpolator 10a designed for video format con- 
version to a lower frame frequency is provided. This 
frame interpolator 10a produces three kinds of interpo- 

55 lated frames FLa, FLb. and FLc from a first video signal. 
It employs four frame memories Ml.-fo.M4', which are 
connected in series to store four past frames of the first 
video signal beingj^upplied from an external source. As 
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FIG. 3 shows, the symbol for. the present incoming 
frame is "M1 in," while those for the outputs of the frame 
memories M1, M2, M3, and M4 are M M2out," "M3out," 
"M4out," and "MSout," respectively. 
[0028] A motion detector MC produces forward-ref- 
erence motion vectors by making a comparison 
between the frames M2out and M3out. It also produces 
backward-reference motion vectors by making a com- 
parison between the frames M1in and M2out. FIG. 4 
explains the concept of forward-reference and back- 
ward-reference motion vectors. Suppose here that the 
frame M2out is now being subjected to the motion 
detection process (i.e:, M2out is the current picture). 
When observed from this frame M2out, the frame M3out 
is regarded as its previous frame, and the frame M 1 in as 
its future frame. A forward -reference motion vector Vf 
represents the direction and distance of a moving object 
observed between the frames M3out and M2out. On the 
other hand, a backward-reference motion vector Va rep- 
resents those observed between the frames M2out and 
M1in. 

[0029] Referring again to FIG. 3, the frame interpo- 
lator 10a has two delay units D1 and D2 each of which 
provides a one-frame delay to given forward-reference 
and backward-reference motion vectors Vf and Va. That 
is, forward-reference and backward -reference motion 
vectors Vf and Va appearing at the output of the delay 
unit D1 have a delay of one frame interval, with respect 
to those at the output of the motion detector MC, while 
those appearing at the output of the delay unit D2 have 
a delay of two frame intervals. 

[0030] The above-noted motion vector outputs of 
the motion detector MC and delay units D1 and D2 are 
supplied respectively to three multipliers 11a, 11b, and 
11 c for multiplication of vector weighting coefficients CI , 
C2, and C3. Here, the vector weighting coefficients C1 
to C3 denote the mixture ratios of forward-reference 
motion vectors and backward-reference motion vectors. 
By multiplying such coefficients, each multiplier 1 la to 
1 1c calculates a weighted mixture of forward-reference 
motion vectors Vf and backward-reference motion vec- 
tors Va thereby outputting weighted motion vectors MV1 
to MV3. According to those weighted motion vectors 
MV1 to MV3, variable delay units VD1 to VD3 apply 
motion compensation to the frames M3out to MSout, 
respectively. They output the resultant frames FL1 to 
FL3 in phase with the three frames M3out to MSout, 
respectively. 

[0031] Switches SW1 to SW3 select video frames 
according to a motion/still selection command provided 
from an external source/ More specifically, they select 
the outputs of the variable delay units VD1 to VD3 when 
the motion/still selection command indicatesi motion pic- 
ture mode. In turn, they select the outputs of the frame 
memories M2 to M4 when the command indicates still 
picture mode. Picture elements of the selected frames 
gro then supplied to multipliers 12a to 12c for multiplica- 
tion of frame weighting coefficients k1 to k3. Here, the 



coefficients kl to k3 give a mixture ratio of the three 
selected frames. The resultant weighted frame signals 
FL1-m to FL3-m are supplied to adders at the next 
stage for frame summation. 

5 [0032] There are three adders 1 3a to 1 3c to obtain 
three interpolated frames. The first adder 13a produces 
an interpolated frame FLa by adding the last two 
weighted frames FL2-m and FL3-m. The second adder 
13b produces another interpolated frame FLb by adding 

10 all the three weighted frames FL1 -m, FL2-m and FL3-m. 
The third adder 13c produces still another interpolated 
frame FLc by adding the first two weighted frames FL1 - 
m and FL2-m. 

[0033] The next section will now explain a specific 

is structure of a video signal generator designed for video 
format conversion to a lower frame frequency (e.g., 30 
Hz to 25 Hz). FIG. 5 is a block diagram of a video signal 
generator 20a. This video signal generator 20a com- 
prises a switch SW21a, a FIFO 22a, a FIFO controller 

20 24a, and a frame switching controller 27. 

[0034] The frame switching controller 27 comprises 
difference detectors 27a to 27d and a switching control- 
ler 27e. The difference detector 27a detects differences 
between the frames M1in and M2out. The difference 

25 detector 27b detects differences between the frames 
M2out and M3out. The difference detector 27c detects 
differences between the frames M3out and M4out. The 
difference detector 27d detects differences between the 
frames M4out and M5out Examining the detected dif- 

30 ferences, the switching controller 27e finds the maxi- 
mum difference value. This information permits the 
switching controller 27e to determine at which frame 
position the scene change or common format switcho- 
ver has occurred, based on a scene change indication 

35 signal Ds or a common format switching command Df 
supplied from external sources. The switching controller 
27e thus outputs an appropriate switching control com- 
mand SWC to the switch SW21a. 
[0035] The switch SW21a has three input terminals 

40 "a," "b," and "c" to receive interpolated frames FLa to 
FLc, the outputs of the adders 13a to 13c described in 
FIG. 3. The switch SW21a selects either one of the 
three inputs according to the switching control com- 
mand SWC as will be described later. 

45 [0036] The FIFO 22a serves as a buffer memory for 
the interpolated frames being supplied from the switch 
SW21a at the rate of 30 Hz. It controls their output tim- 
ings under the control of the FIFO controller 24a, so that 
interpolated frames will be sent out at 25 Hz. 

50 [0037] The switch SW21a is controlled as follows. 
The switch SW21 a has two modes of operation: normal 
mode and transitional mode. In normal mode, the switch 
SW21a keeps its contact sw-1a at the second position 
"b," and thus the FIFO 22a receives interpolated frames 

55 FLb at a rate of 30 Hz. The FIFO 22a decimates some 
of the received frames at predetermined intervals, 
thereby converting the frame frequency down to 25 Hz. 
[0038] Transitional moderas opposed to the normal 
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mode, is selected at scene changes, as well as in the 
case that the common video format is switched from 
CIF to QCIF or vise versa (i.e., common format switcho- 
ver). While later explanation assumes the occurrence of 
a scene change, those skilled in the art will appreciate 
that the same will apply to the operation in the case of 
common format switchover. 

[0039] FIG. 6 shows inputs and outputs of the frame 
memories M1 to M4 (FIG. 3). The symbol "M1 in" is used 
to represent a source frame being entered to the first 
frame memory Ml, while **M2out N to "MSouT represent 
the output pictures of the frame memories M1 to M4, 
respectively. Those five consecutive pictures represent 
a part of a given frame sequence. FIG. 6 shows three 
instances of such frame snapshots taken at times to, t1 , 
and t2, and in this progression, the pictures are shifted 
by one frame toward the right-hand side of FIG. 6. 
[0040] This example video sequence of FIG. 6 
includes a scene change, which actually happened at 
some time before to. The switching controller 27e is 
designed to initiate transitional mode operation when 
such a scene change is observed between the frames 
M3out and M4out. Here, the switching control command 
SWC serves as a message to inform the switch SW21a 
that a scene change has reached that critical point. This 
message- makes the switch SW21a enter the transi- 
tional mode and handle the scene change at times to 
and t1. That is, the transitional mode operation lasts 
only two frame intervals, and the switch SW21a returns 
to normal mode at time t2 to resume normal switching 
operations. 

[0041] At time to (i.e., the first half of transitional 
mode operation), the switch SW21a first has to confine 
the source of interpolated frames to frames captured 
before the occurrence of the scene change. To this end, 
the switch SW21a moves its contact sw-1a to the first 
position "a" according to the switching control command 
SWC, thereby selecting an interpolated frame FLa 
derived from the frames M4out and MSout (i.e., old 
scene). After that, at time tl, the switch SW21a moves 
the contact sw-1a to the third position "c" so as to select 
interpolated frames derived only from a new scene. 
Accordingly, an interpolated frame FLc produced from 
the frames M3out and M4out is selected. At time t2, the 
switch SW21 a resumes its normal mode operation by 
returning its contact sw-1a to the second position "b," 
again selecting interpolated frames FLb being produced 
from M3out, M4out, and M5out. 

[0042] The switch SW21a is designed to change 
the position of its contact sw-1a step by step (i.e., "b"- 
•V-V-V) according to each switching control com- 
mand SWC. However, it is not intended that the inven- 
tion be limited to this structure, but the switch SW21a 
can also be configured to do the same autonomously at 
predetermined intervals, once it receives the first 
switching control command SWC. 
[0043] The next section will how describe a process 
of producing interpolated frames for video format con- 
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version to a lower frame frequency (e.g., 30 Hz to 25 
Hz). 

[0044] „ FIGS. 7 and 8 present a frame interpolation 
process including no scene change or common format 

5 switchover. The symbol Tin" represents a source frame 
sequence having a frame frequency of 30 Hz that is 
given to the frame interpolator 10a. For the sake of con- 
venience, source frames are labeled TV to M F6" in a 
cyclic manner. Consider that the first instance of "F1" 

10 appears at the output of the fourth frame memory M4. 
Then the frames F2, F3, and F4 are at the outputs of the 
frame memories M3, M2, and Ml, respectively, and F5 
at the input of the first frame memory Ml. At the next 
cycle, the frame F2 appears at the output of the frame 

15 memory M4, since the frame memory contents are 
shifted forward by one frame. Likewise, the frames F3. 
F4, and F5 are now at the outputs of the frame memo- 
ries M3, M2, and M1 , respectively, and F6 at the input of 
the first frame memory M1 . 

20 [0045] On the other hand, the symbol "Fout" shows 
a series of interpolated frames FLb produced by the 
video signal generator 20a when the switch SW21 a is in 
normal mode (i.e., its contact is at the second position 
,, b w ). The interpolated frames Fout are labeled "F1a" to 

25 "F5a M in a cyclic manner 

[0046] In the process shown in FIGS. 7 and 8. inter- 
polated frames are produced from the following combi- 
nations of source frames: 

30 * F1 a produced from F1 , F2, and F3 
F2a produced from F2, F3, and F4 
F3a produced from F3, F4, and F5 
F4a produced from F5, F6, and F1 
F5a produced from F6, F1 , and F2 

35 

Note here that, although the frame interpolator 10a pro- 
duces interpolated frames from frames F4 to F6, the 
FIFO 22a in the video signal generator 20a will never 
output them. That is, the proposed video format con- 
40 verter subsamples the produced interpolated frames to 
reduce its frame frequency. 

[0047] FIG. 9 depicts the above-noted relationships 
between source frames (30 Hz) and subsampled 
frames (25 Hz). Each bold solid line represents a frame, 

45 while broken lines indicate unit time intervals. The inter- 
polated frames Fout shown in FIGS. 7 and 8 are fed to 
the FIFO 22a at the rate of 30 Hz. The FIFO 22a regu- 
lates the intervals of output frames so that they will be 
sent out at 25 Hz. 

so [0048] A second embodiment of the present inven- 
tion will now be described below, with reference to 
FIGS. 10 to 17. As opposed to the first embodiment, a 
video format converter proposed of the second embod- 
iment is designed to raise the frame frequency. For illus- 

55 trative purposes, it is assumed here that the proposed 
converter makes a frame frequency conversion from 25 
u-» *~ on u-? 

[0049] Referring first to a blodcdiagram of FIG. 1 0, 
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a frame interpolator 10b designed to raise the frame f re- interpolated frames, FLd. from the frame interpolator 
quency will be described below. This frame interpolator 10b. Note that those interpolated frames are generated 
10b produces four kinds of interpolated frames FLa. at 30 Hz, i.e., the rate of source video frames. The FIFO 
FLb, FLc, and FLd from a first video signal. This frame controller 24b supplies control signals to the two FIFOs 
interpolator 1 0b differs from the frame interpolator 1 0a 5 22b and 25b to regulate the flow of outgoing frames, so 
of FIG. 3 in that a switch SW4 and an adder 13d are that they will be output at 25 Hz. 
newly employed. Since the other functional elements of [0054] The outputs of the two FIFOs 22b and 25b 
the frame interpolator 10b are common to those of the are then supplied to two input terminals "d" and n e n of 
frame interpolator 10a, the following explanation will the second switch SW22b, respectively. According to 
focus on their differences, while maintaining like refer- 10 frame switching commands Dc from the controller 26b, 
ence numerals to like elements. the second switch SW2lb chooses one of the two input 
[0050] The adder 13d produces a fourth interpo- signals as final output of the video signal generator 20b. 
lated frame FLd by adding the weighted frames FL1-m [0055] Referring to FIGS. 10 to 12, the following 
and FL2-m. The switch SW4 is controlled through a section will describe how the switches SW4, SW21b, 
switching control command SWCa. While being nor- is and SW22b operate in the second embodiment. As in 
mally closed, this switch SW4 is turned off when a the first embodiment, those switches have two modes of 
scene change or common format switchover is operation: normal mode and transitional mode. The lat- 
detected, in which case the interpolated frame FLd is ter mode is selected in the case of scene changes or 
identical to the frame FL1-m, since the adder 13d common format switchover. Although the case of a 
becomes transparent. More details of the switch SW4 20 scene change will be discussed later, those skilled in 
and switching control command SWCa will be provided the art will appreciate that the same will apply to corn- 
later, mon format switchover. 

[0051] Referring next to FIG. 11. a video signal gen- [0056] FIG. 12 shows inputs and outputs of the 

erator 20b designed to raise the frame frequency will be frame memories M1 to M4 (FIG. 10). The symbol M1in 

described below. This video signal generator 20b com- 25 is used to represent a source frame being entered to the 

prises two switches SW21b and SW22b. two FIFOs 22b first frame memory M1, while M2out to M5out represent 

and 25b. a FIFO controller 24b. a first frame switching the output pictures of the frame memories M1 to M4, 

controller 27-1 . and a second frame switching controller respectively. These five consecutive pictures represent 

26b. a part of a given frame sequence. FIG. 12 shows four 

[0052] The first frame switching controller 27-1 has 30 instances of such frame snapshots taken at times to, t1 , 

difference detectors 27a to 27d and a switching control- t2. and t3, and in this progression, the pictures are 

ler 27e-1. The difference detector 27a detects differ- shifted by one frame, toward the right-hand side of FIG. 

ences between the frames M1in and M2out. The 12. 

difference detector 27b detects differences between the [0057] In normal mode, the SW4 is closed and the 
frames M2out and M3out. The difference detector 27c 35 first switch SW21b keeps its contact sw-1b at the sec- 
detects differences between the frames M3out and ond position "b." On the other hand, the second switch 
M4out. The difference detector 27d detects differences SW22b connects its contact sw-2b at the first position 
between the frames M4out and M5out. By examining M d H according to the frame switching command Dc, 
the detected differences, the switching controller 27e-1 which allows the produced frames to be transmitted 
finds the maximum difference value. This information 40 through the First FIFO 22b and second switch SW21b in 
permits the switching controller 27e-1 to determine at the following sequence: 
which frame position the scene change or common for- 
mat switchover has occurred, based on a scene change • FLb-1 produced from frames M3out to MSout at 
indication signal Ds or a common format switching com- time to 

mand Df supplied from external sources. The switching 45 • FLb-2 produced from frames M3out to MSout at 

controller 27e-1 thus outputs an appropriate switching timetl 

control command SWC to control the first switch • FLb-3 produced from frames M3out to M5out at 

SW21b. It further produces another switching control timet2 

command SWCa to control the switch SW4 in the frame • FLb-4 produced from frames M3out to M5out at 
interpolator 10b. so timetS 
[0053] The first switch SW2 1b has three input ter- 
minals "a," "b," and "c" to receive three kinds of interpo- [0058] When the interpolated frame FLb-3 is out- 
lated frames FLa, FLb, and FLc, the outputs of the put. the adder 13d produces another interpolated frame 
adders 1 3a to 1 3c shown in FIG. 1 0. This first switch FLd from the frames M3out and M4out and supplies it to 
SW21b selects either one of the three inputs according 55 the second FIFO 25b. After the interpolated frame FLb- 
to the switching control command SWC. and sends the 4 is sent out. the frame switching controller 26b alters its 
selected frames to the first FIFO 22b. On the other command output Dc to make the first FIFO 22b move its 
hand, the second FIFO 25b receives another kind of contact sw-2bto the position-- "e" so as to select the 
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frame FLd held in the second FIFO 25b. The frame 
switching controller 26b then commands the second 
switch SW21b to return its contact sw-2b to the position 
"d," thus repeating the above-described operation. 
[0059] If a scene change is encountered, the pro- s 
posed video format converter enters to transitional 
mode. The operation in this mode will be described 
below with reference to FIGS. 6, 10, and 11. 
[0060] As described earlier, the example video 
sequence of FIG. 6 includes a scene change. The 10 
switching controller 27e-1 (FIG. 1 1 ) is designed to initi- 
ate transitional mode operation, when such a scene 
change is observed between the frames M3out and 
M4out Here, the switching control command SWC 
serves as a message to inform the first switch SW2 1b 15 
that the scene change has reached that critical point. 
This message makes the first switch SW21b enter the 
transitional mode and handle the scene change at times 
to and t1 . The first switch SW21 b then returns to normal 
mode at time t2 to resume normal switching operations. 20 
[0061] At time tO, the first switch SW21b has to con- 
fine the source of interpolated frames to frames cap- 
tured before the occurrence of the scene change. To 
this end, the first switch SW21 b moves its contact sw-1 b 
to the first position "a" according to the switching control 25 
command SWC. thereby selecting an interpolated 
frame FLa produced from the frames M4out and MSout. 
At the same time, the second switch SW22b sets its 
contact sw-2b to the first position "d" according to the 
frame switching command Dc. As a result, the interpo- 30 
lated frame FLa is sent out. 

[0062] At time t1 the first switch SW21b has to 
select an interpolated frame produced only from the 
new scene. The first switch SW21b thus moves its con- 
tact sw-1b to the third position "c," thereby selecting an 35 
interpolated frame FLc produced from the frames M3out 
and M4out. The second switch SW22b maintains its 
contact position "d" according to the frame switching 
command Dc, thus sending out the selected interpo- 
lated frame FLc. 40 
[0063] At time t2, the first switch SW21b returns its 
contact sw-1 b to the second position Td" to resume nor- 
mal mode operation, again selecting interpolated 
frames FLb. The second switch SW22b keeps its con- 
tact position "d" according to the frame switching com- 45 
mand Dc, thus sending out the selected interpolated 
frame FLb. 

[0064] As discussed above, the first switch SW21b 
is designed to change the position of its contact sw-1b 
step by step (i.e., "b^c"-^"-"^) according to each 50 
switching control command SWC. However, it is not 
intended that the invention be limited to this structure, 
but the first switch SW21b can also be configured to do 
the same autonomously at predetermined intervals, 
once it receives the first switching control command 55 
SWC. * 

[0065] FIG. 13 shows such a situation where the r 
proposed converter encounters a scene change when 



producing an interpolated frame FLd in normal mode. 
Although it is normally closed, the switch SW4 is 
designed to become open when a scene change is 
observed at a particular point shown in FIG. 13. This 
means that the aforementioned switching control com- 
mand SWCa serves as a message to inform the switch 
SW4 that a scene change has reached that critical 
point. With the switch SW4 turned off, the adder 13d 
produces an interpolated frame FLd solely from the 
frame M3out. 

[0066] Referring now to FIGS. 14 and 15, the next 
section will describe a process of producing interpo- 
lated frames to convert the video format to a higher 
frame frequency (e.g., 25 Hz to 30 Hz). This illustrated 
process, however, includes no scene change or com- 
mon format switchover. 

[0067] The symbol "Fin" represents a source frame 
sequence having a frame frequency of 25 Hz that is 
given to the frame interpolator 1 0b. For the sake of con- 
venience, source frames are labeled "F1" to "F5 W in a 
cyclic manner. Consider that the first instance of f, F1" 
appears at the output of the fourth frame memory M4. 
Then the frames F2, F3, and F4 are at the outputs of the 
frame memories M3, M2, and M1 , respectively, and F5 
at the input of the first frame memory M 1 . 
[0068] At the next cycle, the frame F2 appears at 
the output of the frame memory M4, since the frame 
memory contents are shifted forward by one frame. 
Likewise, other frames F3, F4, and F5 are now at the 
outputs of the frame memories M3, M2, and M1, 
respectively, and the second instance of F1 appears at 
the input of the first frame memory M1 . 
[0069] On the other hand, the symbol "Foutl" 
shows a series of interpolated frames FLb produced by 
the video signal generator 20b when the first switch 
SW21b is in normal mode (i.e., its contact is at the sec- 
ond position "b"). Those frames Foutl are labeled 
"F1a." "F2a," "F3a," n F5a," and "F6a" in a cyclic manner. 
Further, the symbol "Fout2" shows additional interpo- 
lated frames FLd produced by the adder 13d in the 
frame interpolator 10b. In FIGS. 14 and 15, such inter- 
polated frames are labeled "F4a." 
[0070] The above process as a whole generates six 
interpolated frames in the following sequence during 
five source frame intervals: 

F1 a produced from F1 , F2, and F3 
F2a produced from F2, F3 f and F4 
F3a produced from F3, F4, and F5 
F4a produced from F4 and F5 
F5a produced from F4, F5, and F1 
F6a produced from F5, F1 , and F2 

Note here that the frames F3a and F4a are produced at 
the same time. The flow of produced frames is regu- 
lated by controlling the FIFOs, thus obtaining final out- 



[0071] FIG. 1 ^depicts the reTationships between 
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source frame sequence (25 Hz) and converted frame the invention to the exact construction and applications 
sequence (30 Hz). Each bold solid line represents a shown and described, and accordingly, all suitable mod- 
frame, while broken lines indicate unit time intervals. ifications and equivalents may be regarded as falling 
The interpolated frames Foutl and Fout2 shown in within the scope of the invention in the appended claims 
FIGS. 14 and 1 5 are fed to the FIFOs 22b and 25b for 5 and their equivalents, 
output flow control. The FIFOs 22b and 25b serve as 
buffer storage to regulate the intervals of output frames Claims 
so that they will be sent out at 30 Hz. 

[0072] As described above, the video format con- 1. An apparatus for converting from a first video signal 

verter 1 of the present invention uses frame irrterpola- 10 in a first video format to a second video signal in a 

tion techniques to obtain a new frame frequency. That second video format that is incompatible with the 

is, the video format converter 1 is designed to adaptively first video format, comprising: 
generate interpolated frames from the first video signal 

by considering motion vectors obtained therefrom. A frame interpolating means for producing inter- 
second video signal is produced from the interpolated is polated frames from the first video signal by 
frames, so that it will have a different frame frequency. using motion vectors obtained therefrom; and 
Accordingly, the video format converter 1 of the present video signal generating means for producing 
invention provides smooth motion in a scene, eliminat- the second video signal from the interpolated 
ing discontinuity in a frame sequence, which could be frames, 
introduced in the process of frame frequency conver- 20 

S i 0n 2. The apparatus according to claim 1 , wherein said 
[0073] The following section will now describe a frame interpolating means produces a plurality of 
video format conversion method according to the frames by using the motion vectors for motion corn- 
present invention. FIG. 17 is a flowchart showing a pensation, multiplies the produced frames by 
process to cdnvert from a first video signal in a first 25 weighting coefficients, obtains weighted sums by 
video format to a second video signal in a second video adding the weighted frames, and outputs the 
format that is incompatible with the first video format weighted sums as the interpolated frames. 
The process comprises the following steps: 

3. The apparatus according to claim 1 . wherein said 

(51) Interpolated frames are produced from the first 30 frame interpolating means uses forward-reference 
video signal by using motion vectors of the first motion vectors and backward-reference motion 
video signal. vectors as the motion vectors. 

(52) The second video signal is produced from the 

interpolated frames. 4. The apparatus according to claim 1 , wherein said 

35 frame interpolating means produces the interpo- 

[0074] More specifically, the difference between the lated frames, while switching between still picture 

two video formats lies in their frame frequencies. That mode and motion picture mode, 
is, the proposed method performs frame interpolation of 

the first video signal having a first frame rate, thereby 5. The apparatus according to claim 1 . wherein: 

generating the second video signal having a second 40 

frame frequency. the first video format has a first frame fre- 

[0075] The above-described embodiments of the quency; 

present invention is now summarized as follows. the second video format has a second frame 

According to the present invention, a method and appa- frequency that is different from the first frame 

ratus for converting between different video formats are 45 frequency; and 

proposed. They produce interpolated frames from a first said video generating means produces the 

video signal, and from these interpolated frames, gener- second video signal having the second frame 

ate a second video signal that is incompatible with the frequency, 
first video signal. With appropriate frame interpolation 

techniques, the present invention promises smooth so 6. The apparatus according to claim 1 , wherein: 
video motion in visual communications between remote 
users of different television systems, eliminating unnat- 
ural effects which could be introduced in the process of 
video format conversion. 

[0076] The foregoing is considered as illustrative 55 
only of the principles of the present invention. Further, 
since numerous modifications and changes will readily 
occur to those skilled in the art, it is not desired to limit 
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said video generating means detects differ- 
ences between consecutive frames of the first 
video signal; and 

when a scene change is indicated, said video 
generating means produces the second video 
signal by choosing, based on the detected dif- 
ferences, such interpolated frames that are 
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derived from either group of correlated source 
frames, those before the scene change or 
those after the scene change, in the first video 
signal. 

7. The apparatus according to claim 1, wherein: 

said video generating means detects differ- 
ences between consecutive frames of the first 
video signal; and 

when common format switchover is indicated, 
said video generating means produces the 
second video signal by choosing, based on the 
detected differences, such interpolated frames 
that are derived from either group of correlated 
source frames, those before the common for- 
mat switchover or those after the common for- 
mat switchover, in the first video signal. 

8. A method of converting from a first video signal in a 20 
first video format to a second video signal in a sec- 
ond video format that is incompatible with the first 
video format, comprising the steps of: 



10 



15 



(a) producing interpolated frames from the first 25 
video signal by using motion vectors obtained 
therefrom: and 

(b) producing the second video signal from the 
interpolated frames. 

30 

9. The method according to claim 8, wherein: 

the first video format has a first frame fre- 
quency; and 

the second video format has a second frame 35 
frequency that is different from the first frame 
frequency. 
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video motion by eliminating unnatural effects which 
could be introduced in the process of video format con- 
version. A frame interpolator produces interpolated 



frames from a first video signal given in a first video for- 
mat by using motion vectors of the first video signal. 
From the interpolated frames, a video signal generator 
produces a second video signal in a second video for- 
mat that is incompatible with the first video format. 
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